Method and system for detecting objects of interest

ABSTRACT

A method for detecting objects ( 28 ) of interest comprises: receiving gaze tracking data ( 30 ) of a person looking at a collection of objects ( 28 ), the gaze tracking data ( 30 ) comprising information, at which points the person is looking; determining a map ( 42 ) of objects in the field of view of the person, the map ( 42 ) of objects indicating, at which position ( 44 ) an object in the collection of objects is arranged; determining gaze interaction events ( 48 ) for objects in the collection of objects by matching the gaze tracking data ( 30 ) to the map ( 42 ) of objects, wherein a gaze interaction event ( 48 ) for an object indicates that a specific gazing interaction has occurred with respect to the object; determining category characteristics ( 56 ) for at least those objects ( 28 ) having gaze interaction events ( 48 ), wherein a category characteristic ( 56 ) describes the object with respect to a specific object category ( 58 ); and determining at least one category characteristic ( 56 ) of interest in at least one object category ( 58 ) by finding at least one category characteristic ( 56 ) which is assigned to a plurality of objects ( 28 ), which have gaze interaction events ( 48 ) indicating the person is interested in these objects ( 28 ).

REFERENCE TO RELATED APPLICATIONS

This application claims the benefit and filing date of European patentapplication EP 14191120.6 filed 30 Oct. 2014.

FIELD OF THE INVENTION

The invention relates to a method, a computer program and acomputer-readable medium for detecting objects of interest with a gazetracking system. Furthermore, the invention relates to a gaze trackingsystem.

BACKGROUND OF THE INVENTION

Devices like PCs, laptops, smartphones or head mounted eye trackers(such as augmented reality glasses, virtual reality glasses, eyetracking glasses or smart glasses) may be equipped with a gaze trackingsystem, which is adapted for determining the direction in which a personusing the device is looking (eye tracking) and for determining, at whichpoint the person is looking (gaze tracking).

A gaze tracking system may be used for supporting a search, which is akey activity when working with computers. However, optical searches ofhumans are usually happening both in the focus point as well as in theparafoveal, perifoveal and peripheral view, summarized in the followingby the term peripheral view. When searching for a graphicaldistinguishable object (such as an image, a word, a text block, an icon,etc.), the focus point of human gaze is selectively directed to objectsof interest it pre-identifies in the peripheral view based on opticalrecognizable aspects or object categories (e.g. dominant colours,structure, specific elements like people, text or graphics in images andicons, word length, arrangement of small, high rounded and straightletters in words) as potential candidates. The category characteristicswhich may be detected by a human in his peripheral view may depend onthe search task itself and are generally limited by the capabilities ofhuman vision in different areas of the peripheral view.

DESCRIPTION OF THE INVENTION

It is an object of the invention to support a computer-based search witha gaze tracking system in an efficient way.

This object is achieved by the subject-matter of the independent claims.Further exemplary embodiments are evident from the dependent claims andthe following description.

An aspect of the invention relates to a method for detecting objects ofinterest, which completely may be performed by a computer. Objects ofinterests may be images and/or words displayed on a screen provided by acomputer or may be real world objects, which for example are watched bya person through smart glasses or a head mounted eye tracker.

According to an embodiment of the invention, the method comprises:receiving gaze tracking data of a person looking at a collection ofobjects, the gaze tracking data comprising information, at which pointsthe person is looking; determining a map of objects in the field of viewof the person, the map of objects indicating, at which position anobject in the collection of objects is arranged; determining (at leastone type of) gaze interaction events for objects in the collection ofobjects by matching the gaze tracking data to the map of objects,wherein a gaze interaction event for an object indicates that a specificgazing interaction has occurred with respect to the object; determiningcategory characteristics for objects having gaze interaction events,wherein a category characteristic describes the object with respect to aspecific object category; and determining at least one categorycharacteristic of interest in at least one object category by finding atleast one category characteristic which is assigned to a plurality ofobjects, which have gaze interaction events indicating the person isinterested in these objects.

For example, the gaze tracking data may comprise data points, which maycomprise two coordinates, and which indicate a current focus point ofone or two eyes of the person. To determine at which objects the personis looking, the map of objects may comprise the outlines or shapes ofthe objects. For example, the map of objects may comprise an outliningrectangle or in general an outlining polygon or outlining shape for eachobject in the collection of objects.

The data points of the gaze tracking data then may be mapped against themap of objects, for example by determining, which data points arepositioned inside which outlines of the objects. Furthermore, gazeinteraction events may be identified from the gaze tracking data, forexample, by counting the number of gaze data points, which arepositioned inside the respective outline of an object. It also may bepossible to derive motion vectors from the data points (a vector havinga support point and a direction), which give rise to further gazeinteraction events, as, for example, predicted target points and/or acrossing of an object with a gaze vector.

At least for some of the objects, for which gaze interaction events havebeen identified, category characteristics are determined. For example,these category characteristics may be a specific color, a specific shapeand/or a specific pattern in an image, when the objects are images orreal world objects. Category characteristics may be assigned to objectcategories such as color, shape and/or a pattern. It has to beunderstood that only category characteristics of one or more specialtype of object categories may be considered (such as color and/orshape).

In the end, at least one category characteristic of interest in at leastone object category is determined by combining the informationdetermined during the previous steps, i.e. gaze interaction eventsassigned to objects and category characteristics assigned to objects. Ingeneral, by statistical evaluating the gathered information, one or morecategory characteristics are determined by finding one or more groups ofsimilar category characteristics, which all are assigned to a pluralityof objects, which are interesting for the person.

For example, the category characteristics of a specific category (suchas color) may be represented by a single value (for example a specificcolor value) or a plurality of values (for example averaged color valuesspread over a grid, which is overlaying an image) and two categorycharacteristics may be similar, when their value(s) differ from eachother only at least by a threshold. A statistical function (such asleast mean square), which is applied to (the value(s) of) a categorycharacteristic may be used for determining a statistical value, whichmay be compared to a mean value for deciding, whether the categorycharacteristic is in a specific group represented by the mean value.

In one case, an object may be declared interesting for a person, whenthe object is assigned to a gaze interaction event. As an example, agaze interaction event indicates that the person is interested in anobject, if the gaze interaction event is based on gaze tracking datahaving data points in a map area of the object. It also may be possiblethat a grade of interest or involvement level may be represented by avalue that may be determined with a statistical function (for exampledependent on gaze interaction events) and that an object is declaredinteresting, when its involvement level value is higher than a thresholdvalue.

According to an embodiment of the invention, category characteristics ofobjects are clustered and the at least one category characteristic ofinterest is determined by selecting at least one cluster of categorycharacteristics representing the at least one category characteristic ofinterest. The category characteristics may be clustered with respect toobject categories and/or with respect to one or more mean categorycharacteristics that may be determined with a statistical function. Forexample, in the category of average colors, the statistical function mayfind a cluster of blue average colors (which may comprise different bluecolor values) and a cluster of red average colors).

According to an embodiment of the invention, at least two differenttypes of gaze interaction events are determined for an object. It has tobe understood that from the pure gaze tracking data (i.e. data points),more complex gaze interaction events may be derived. For example, a gazeinteraction event may be based on at least one of: gaze hits on theobject, a duration of a gaze focus on the object, a fixation detectionwith respect to the object, a reading pattern detection with respect tothe object, revisits of an object, detail scanning pattern, interactiontime after a first appearance of an object on a display/in a peripheralview, a length of visit time, a time after finished loading of content,etc.

According to an embodiment of the invention, a gaze interaction event isbased on a model of a peripheral view of the person. Usually, the gazetracking data comprises data points which indicate the focus point ofthe gaze of the person. However, a person looking for specificcharacteristics during a search also uses his or her peripheral view fordetermining, which objects are interesting and which not. The peripheralview of a person may be defined as the view outside of the focus pointsor focus area.

In one case, a peripheral view may modeled with an area surrounding afocus point and excluding the focus point. The peripheral view may bemodeled with one or more ring areas surrounding the focus point. Anobject, which map area overlaps with the peripheral view of a gazeinteraction event, may be declared as interesting or as not interesting.A grade of interest value of an object may be based on a value assignedto such a gaze interaction event.

Gaze interaction events considering the peripheral view may be based onat least one of: a duration of the object in a peripheral view of theperson, a level of involvement with the peripheral view of the person,outliers pointing at a specific object but that do not result in a focuspoint on the specific object, etc.

According to an embodiment of the invention, category characteristicsfor at least two different types of object categories are determined foran object. It has to be understood that not only one but two or moreobject categories are evaluated simultaneously for the objects ofinterest. It also may be possible that interesting object categories aredetermined during the evaluation of the gaze tracking data and that thedetermination of category characteristics is restricted to these objectcategories.

According to an embodiment of the invention, the categorycharacteristics for the collection of objects are precalculated andstored together with the objects. For example, the objects may be storedin a database (such as images or videos) and the database also maycontain category characteristics for a number of categories that havebeen calculated during or before the composition of the database. In thecase of images and videos, for example, an average color value may becalculated and stored together with the respective multimedia data ofthe image or video.

According to an embodiment of the invention, the categorycharacteristics are determined by evaluating object data of the objects.For example, the object data such as image data, video data or text datamay be evaluated “on the fly”, i.e. during or after the evaluation ofthe gaze tracking data, when it is displayed on a display device and/orduring or after recording with a camera of the gaze tracking system.

According to an embodiment of the invention, object categories aredetermined by evaluating object data of objects from the group ofinteresting objects. It also may be possible that the object data isevaluated with a statistical method or with a machine learning method togenerate object categories.

According to an embodiment of the invention, the method furthercomprises: determining an involvement level for an object from the gazeinteraction events (and optionally the category characteristics) of theobject, wherein only objects with an involvement level higher than aninvolvement level threshold are considered for the identification ofinterest, selection of objects and/or the selection of objectcategories. The involvement level may be a numerical value that iscalculated based on the gaze interaction events. For example, differentgaze interaction events (such as a fixation point or object beingtouched by a movement vector) may be assigned to different, for examplepredefined, involvement level values and the involvement level of anobject is based on the involvement level values of the gaze interactionevents (which, for example, may be multiplied or added).

Additionally, it may be possible that the involvement level may be basedon common interests with other persons. For example, when a number ofother users or persons find category characteristics a and binteresting, for a using finding characteristic a interesting, theinvolvement level for objects having characteristic b may be increased.

According to an embodiment of the invention, the method furthercomprises: determining a negative involvement level for an object fromthe gaze interaction events of the object (in particular based on amodel of a peripheral view of the person) (and optionally the categorycharacteristics of the object), wherein objects with a negativeinvolvement level higher than a negative involvement level threshold areexcluded for the selection of objects. The negative involvement levelmay be calculated analogously to the positive involvement level.

As an example, a person explicitly not looking at a specific object mayhave seen in his or her peripheral view that the object does not havethe desired characteristic (because it may have the wrong color). Inthis case, the gaze interaction event “object touches peripheral view”without the gaze interaction event “focus” may result in a high negativeinvolvement level.

The positive and negative involvement level thresholds may be predefinedor dynamically determined. In the latter case, the values of theinvolvement level may be sorted and a cutoff may be determined where thecurve of ascending or descending values has a steep flank (thederivative of the curve reaches a specific value). In general, thecutoff may be determined with any type of clustering.

According to an embodiment of the invention, the collection of objectsis displayed on a display device in the field of view of the person. Forexample, the gaze tracking device monitors a person looking at themonitor of a computer, which displays the objects and also performs themethod for detecting which objects, category characteristics and/orcategories the person has interest in.

According to an embodiment of the invention, after the at least onecategory characteristic of interest has been determined, additionalobjects having the at least one characteristic of interest are displayedon the display device. In the case, the objects are displayed on amonitor of the display device, it also may be possible that the computerperforming the method offers the person more objects that have thecharacteristics which seem to be interesting for the person.

According to an embodiment of the invention, the map of objects isgenerated from outlines of objects. In the case, the objects aredisplayed by the device, which also performs the method, outlines orshapes of the objects to be displayed may be determined and may be usedfor mapping the gaze tracking data to the objects. However, it also maybe possible that the map of objects is generated from image datareceived from a camera recording the field of view of the person.

According to an embodiment of the invention, the collection of objectscomprises at least one of: images, words, text blocks, icons, formulas,pages, sections of a document, 3D objects, shapes, patterns. All thesetypes of objects may have different types of object categories, such ascolors, shapes, word types, patterns, etc.

According to an embodiment of the invention, the object categories arevisual descriptors, wherein the category characteristics are values ofcontent descriptors. Several standards like the MPEP-7 standard definesstandardized content descriptors, which associate values tocharacteristics of, for example, multimedia data. In the case of MPEP-7examples for these descriptors are the Scalable Color Descriptor, theColor Layout Descriptor, the Dominant Colors Descriptor, and the EdgeHistogram Descriptor.

Further aspects of the invention relate to a computer program fordetecting objects of interest, which, when being executed by aprocessor, is adapted to carry out the steps of the method as describedin the above and in the following, and to a computer-readable medium, inwhich such a computer program is stored.

A computer-readable medium may be a floppy disk, a hard disk, an USB(Universal Serial Bus) storage device, a RAM (Random Access Memory), aROM (Read Only Memory), an EPROM (Erasable Programmable Read OnlyMemory) or a FLASH memory. A computer-readable medium may also be a datacommunication network, e.g. the Internet, which allows downloading aprogram code. In general, the computer-readable medium may be anon-transitory or transitory medium.

A further aspect of the invention relates to a gaze tracking system,which, for example, may be provided by a laptop or by a head mounted eyetracker. It has to be understood that features of the method, thecomputer program and the computer-readable medium as described in theabove and in the following may be features of the gaze tracking systemas described in the above and in the following, and vice versa.

According to an embodiment of the invention, the gaze tracking systemcomprises a gaze tracking device for generating gaze tracking data andan object detection device, which may be adapted for performing themethod as described in the above and in the following.

The gaze tracking device may comprise a light source for illuminatingthe eyes of a person and a camera or sensor for detecting the movementof the eyes. The gaze tracking device may generate gaze tracking data,for example a sequence of 2D data points, which indicate points a personis looking at.

The object detection device may be part of a computer, a smartphone, atablet computer, or head mounted eye tracker for example attached tosmart glasses. In general, the object detection device may be adaptedfor processing the gaze tracking data.

The gaze tracking system furthermore may comprise a display device fordisplaying the collection of objects, for examples images.

In summary, when working with a computer with gaze tracking andsearching through a high number of objects (e.g. several hundreds tothousands of words, images), the information about focus point andperipheral vision may be used to identify similarities between theobjects of interest. The information may support the user (i.e. personusing the gaze tracking system) actively in her or his search by e.g.displaying a subsample containing the most probably search result forobjects of the total number of objects, adapting the display order basedon the assumed interest or, in a more indirect approach, using thisinformation about similarities possibly combined with the informationabout the final selection to adapt the display of objects to other usersmaking a selection in a similar group of objects after the search of theinitial user.

The method may be used to identify objects or groups of objects ofinterest based on gaze data provided by a gaze tracking system (varioussetups from remote eye tracking to heads up displays with included eyetrackers) to support the user in real time or near real time in opticalbased search processes.

While the movements of the focus point and several other aspects of thegaze interaction events (duration of the focus, distribution of thefocus points, etc.) may reveal information about the objects of interestof the user, at the same time the focus point is attracted to objectsbased on personal interests reaching from repulsion over peculiarities,unfamiliarity and memories to attractiveness as well as objectindependent influences on the search process like being interrupted oridly gazing at the screen. These influences on the search process alsomay be compensated by the method.

These and other aspects of the invention will be apparent from andelucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Below, embodiments of the present invention are described in more detailwith reference to the attached drawings.

FIG. 1 shows a gaze tracking system according to an embodiment of theinvention.

FIG. 2 shows a gaze tracking system according to a further embodiment ofthe invention.

FIG. 3 shows a flow diagram for a method for detection of objects ofinterest according to an embodiment of the invention.

FIG. 4 schematically shows a map of objects used in the method of FIG.3.

FIG. 5 schematically shows a model for a peripheral view used in themethod of FIG. 3.

FIG. 6 schematically shows object categories and characteristics used inthe method of FIG. 3.

The reference symbols used in the drawings, and their meanings, arelisted in summary form in the list of reference symbols. In principle,identical parts are provided with the same reference symbols in thefigures.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

FIG. 1 shows a gaze tracking system 10 in the form of a laptop, whichcomprises a housing 12, a track pad 14 and a keyboard 16 attached to thehousing 12, and a screen or display device 18, which is attached to thehousing 12 via a hinge 20. On the hinge 20, a remote gaze tracker 22 forexample comprising an infrared light source and corresponding sensors (acamera) is provided. Furthermore, a 3D camera 24 is provided above thescreen 18.

With the remote gaze tracker 22 the laptop may be used as a gazetracking device 26. Infrared light from the gaze tracker 22 is shone atthe user and reflections from the eyeballs are recorded with an internalcamera of the gaze tracker 22. From the recorded data of the infraredcamera, the laptop 10 then predicts where the eyes are looking. Inparticular, the laptop 10 generates gaze tracking data, which may beprovided to software components running in the laptop.

As will be explained in more detail below, the laptop 10 may displayobjects 28 and may use the gaze tracking data to detect, which objects28 or which characteristics of the objects are interesting for the user.In this sense, the laptop 10 may be seen as an object detection device30.

FIG. 2 shows a further type of gaze tracking system 10 comprisingglasses 32 as gaze tracking device 26 and an object detection device 30,which may be a PC or smartphone that is communicatively interconnectedwith the glasses 32. The glasses 32 comprise sensors or cameras 34 forrecording the eyes of a wearer of the glasses 32 and the glasses 32 orthe object detection device may generate gaze tracking data from therecorded data of the sensors 34.

The glasses 32 furthermore comprise a camera 36 for recording the fieldof view of the wearer. With the camera data of the camera 36, the objectdetection device 30 may detect objects the person is looking at (forexample by finding outlines or shapes in the camera data) and with thegaze tracking data may detect, which objects or characteristics of theseobjects are interesting for the wearer.

FIG. 3 shows a method for detecting objects of interest, which may beperformed by the object detection device/module 30 of FIG. 1 or FIG. 2.

In step S10, the object detection device 30 receives gaze tracking dataof a person looking at a collection of objects, the gaze tracking datacomprising information, at which points the person is looking. Forexample, the collection of objects may be images on the display 18 ormay be objects in her or his field of view, when wearing the glasses 32.

As shown in FIG. 4, the gaze tracking data 38 may comprise a sequence ofdata points 40 that, for example, may be recorded by the gaze trackingdevice 26 of FIG. 1 or FIG. 2 with a constant or variable sampling rate.The data points 40 may comprise two coordinates that may be mapped tothe coordinates of the display 18 and/or a correction may be applied toadjust the assumed focus point.

In step S12, a map 42 of objects 28 in the field of view of the personis determined, the map 42 of objects indicating, at which position anobject 28 in the collection of objects is arranged.

An example of such a map is also shown in FIG. 4. Each object 28 has amap area 44 in the map 42. In the case of FIG. 4, the map area 44 areequally sized rectangles. For example, the objects 28 may be equallysized images and the map area 44 may be the outer border of theseimages.

When the objects 28 have a more complicated form (for example, when theobjects are part of an image), also outlines 46 may be determined forthe objects 28 and the map 42 of objects may be generated from theseoutlines 46.

In the case, that the objects are real world objects (for example in thecase of FIG. 2) and/or are not displayed by the gaze tracking system 10itself, the map 42 of objects 28 may be generated from image datareceived from the camera 36. For example, an object recognitionalgorithm may be applied to the camera data and outlines of theseobjects are used for generating the map 42.

In step S14, gaze interaction events 48 are determined by matching thegaze tracking data to the map 42 of objects, wherein a gaze interactionevent 48 for an object indicates that a specific gazing interaction hasoccurred with respect to the object 28. The gaze tracking data 30 isprocessed by one or more algorithms, which determine the gazeinteraction events 48 from the gaze tracking data.

These algorithms may be neutrally with respect to a position of theobjects 28 and the detected events 48 may have a position that may bemapped to one or more objects 28. An example for such a gaze interactionevent 48 may be a fixation point 48 a, which may be generated when aspecific number of data points 40 is located within a specific area.

There also may be algorithms already focused on the objects 28. Examplesfor such algorithms are gaze hits 48 b on the object 28 or a duration ofgaze focus on the object 28.

It also may be possible that a gaze interaction event 48, 48 c is basedon a model 50 of a peripheral view of the person as shown in FIG. 5. Theevent 48 c may be described as the event “object only in peripheralview”.

FIG. 5 shows a model 50 of the peripheral view comprising an innercircle around a data point 40 modeling the focus (foveal) view 52 of theperson and an outer ring modeling the peripheral view 54 of the person.In FIG. 5 the peripheral view is modeled with an area surrounding thefocus point and excluding the focus point.

Based on the model 50 (which also may comprise several concentricrings), the gaze tracking data on the peripheral view may be processedby several algorithms, which also may be neutrally with respect toposition or which may be focused on the objects. Examples for suchalgorithms are a duration of an object 28 in the peripheral view, alevel of involvement within the peripheral view, etc.

The model 50 may be adjusted to the person based on information aboutthe person (for example based on monitored gaze/selection behavior).With respect to FIG. 5, the radii of the circle and the ring may beadapted to the person.

In step S16, category characteristics for the objects 28 having gazeinteraction events 38 are determined. As indicated in FIG. 6, everyobject 28 may have one or more category characteristics 56, whichdescribe the object 28 with respect to a specific object category 58.

For example, the object categories 58 may be average color 58 a, colorpattern 58 b, contains head 58 c, shape 58 d, etc. The characteristics56 in these cases may be a color value 56 a, a plurality of (color)values 56 b, a Boolean value 58 c and other data 58 d like “circle”,“square”, “star”).

It may be possible that the category characteristics 56 for the objects26 are precalculated and stored together with the objects 26. Forexample, the characteristics 56 may be (standard) descriptors storedtogether with the objects in a database.

It also may be possible that the category characteristics 56 aredetermined by evaluating object data of the objects 26 on the fly. Inthe case, when relevant categories 58 are not known, upfront and/oraspects of similarities in known object categories 58 are built upduring the search process. For example, the gaze tracking system 10 mayknow that the person is looking at images and considered objectcategories 58 for similarities in images are highlight colors,granularity, persons in image, etc., and these aspects are beinganalyzed in real time or near real time with respect to the objects 26the user is looking at.

Additionally, it may be possible that even object categories 58 (notknown before) are determined by evaluating object data of objects 26. Inthe case, when not even the object categories 58 for similarities areknown, then similarities between BLOBs (binary large objects), i.e. theobject data, may be analyzed based on machine learning algorithms, withor without taking into consideration, which kind of data is encoded intothe object data.

In step S18, one or more category characteristics 56 of interest in oneor more object categories 58 are determined by finding at least onecategory characteristic 56 which is assigned to a plurality of objects28, which have gaze interaction events 48 indicating the person isinterested in these objects.

The indicator, whether an object 28 is of interest for the person or notmay be calculated based on a negative and/or positive involvement level,which may be a numerical value predicting the grade of interest or gradeof disinterest with respect to an object.

Only objects 28 with an involvement level higher than an involvementlevel threshold are considered to be interesting objects and/or objects28 with a negative involvement level higher than a negative involvementlevel threshold are excluded from the selection of interesting objects.

Here, the threshold or cutoff value for the positive and/or negativeinvolvement level may dynamically be adjusted based on algorithms thatdetect clusters or steep drops in the curves of ordered involvementlevel over all objects 26.

For example, specific gaze interaction events 48 may have a predefinedpositive involvement level and/or may have a predefined negativeinvolvement level, and the involvement levels of an object are thenbased on the gaze interaction event specific levels. For example, theinvolvement level of a fixation point 48 a may be higher than that ofnumber of hits 48 b. A gaze interaction event 48 c based on peripheralview may have a negative involvement level. For example, when an object28 that was in a specific area 54 of the peripheral view during a timewhen the peripheral view was assumingly being processed actively but hasnot been focused afterwards.

After that, the category characteristics 56 of the interesting objects28 are clustered and the at least one category characteristic ofinterest is determined by selecting at least one cluster of categorycharacteristics representing the at least one category characteristic ofinterest. FIG. 6 shows two such clusters 60. For example, cluster 60 amay be “objects being blue”, wherein blue may be described by a range ofcolor values that are perceived as blue. As a further example, cluster60 b may be “objects showing a circle”.

As described above, category characteristics 56 of one object category58 may be numerical, integer or Boolean values and/or may becollections/vectors/matrices of those values. These values or sets ofvalues may be evaluated using statistical methods. For example, withleast mean square a mean value may be determined for a specific categorycharacteristic 56 and only those category characteristics 56 which haveonly a distance below a threshold may be taken into a cluster.

The category characteristics 56 may be clustered based on a singleobject category 58.

It may be possible that clustering takes place via several objectcategories 58. For example, the values (or sets of values) categorycharacteristics 56 of several object categories 58 of one object 28 maybe gathered into one vector or matrix and the vectors or matrices ofinteresting objects 28 may be evaluated for finding similar patterns.Such objects 28 with similar patterns may be clustered into one cluster.The category characteristic of interest may then be determined asrepresentative of such a cluster. In this case, similarity may bedefined that a statistical value that may be calculated from the vectoror matrix only differs at least a threshold value from a mean value.

In general, it also may be possible that the clustering takes placebased in the (positive and/or negative) involvement level in combinationwith the category characteristics, i.e. that vectors or matrices areformed from the involvement level and characteristics and the clusteringis based on these vectors or matrices analogously as described above.

In the case, the collection of objects 28 is displayed on a displaydevice 18 in the field of view of the person, in step S20, additionalobjects 28 having the at least one characteristic of interest may bedisplayed on the display device 18.

For example, when trends among category characteristics 56 of interestare detected, this information may be applied to the rest of the objects28 being searched and/or further objects may be pulled from a databaseof similar objects 28 that is accessible by the gaze tracking system 10.

When having a history of the user, information about the priorfamiliarity with a subset of the objects and/or when having someadditional information on the objects, this information also may be usedin the search process. For example, images in a certain date range whensorted by date may be ignored and/or the system 10 may know a group ofimages in detail and is not including these because it knows thesearched item is not included.

While the invention has been illustrated and described in detail in thedrawings and foregoing description, such illustration and descriptionare to be considered illustrative or exemplary and not restrictive; theinvention is not limited to the disclosed embodiments. Other variationsto the disclosed embodiments can be understood and effected by thoseskilled in the art and practicing the claimed invention, from a study ofthe drawings, the disclosure, and the appended claims. In the claims,the word “comprising” does not exclude other elements or steps, and theindefinite article “a” or “an” does not exclude a plurality. A singleprocessor or controller or other unit may fulfil the functions ofseveral items recited in the claims. The mere fact that certain measuresare recited in mutually different dependent claims does not indicatethat a combination of these measures cannot be used to advantage. Anyreference signs in the claims should not be construed as limiting thescope.

LIST OF REFERENCE SYMBOLS

-   10 gaze tracking system-   12 housing-   14 track pad-   16 keyboard-   18 display device-   20 hinge-   22 gaze tracker-   24 3D camera-   26 gaze tracking device-   28 object-   30 object detection device-   32 head mounted eye tracking system-   34 eye tracking sensor-   36 camera-   38 gaze tracking data-   40 data point-   42 map of objects-   44 map area-   46 outline-   48 gaze interaction event-   50 model of peripheral view-   52 focus view-   54 peripheral view-   56 category characteristic-   58 object category-   60 cluster

1. A method for detecting objects of interest, the method comprising:receiving gaze tracking data of a person looking at a collection ofobjects, the gaze tracking data comprising information, at which pointsthe person is looking; determining a map of objects in the field of viewof the person, the map of objects indicating, at which position anobject in the collection of objects is arranged; determining gazeinteraction events for objects in the collection of objects by matchingthe gaze tracking data to the map of objects, wherein a gaze interactionevent for an object indicates that a specific gazing interaction hasoccurred with respect to the object and wherein a gaze interaction eventis based on a model of a peripheral view of the person; determiningcategory characteristics for objects having gaze interaction events,wherein a category characteristic describes the object with respect to aspecific object category; determining at least one categorycharacteristic of interest in at least one object category by finding atleast one category characteristic which is assigned to a plurality ofobjects, which have gaze interaction events indicating the person isinterested in these objects determining a negative involvement level foran object from the gaze interaction events of the object based on themodel of the peripheral view of the person, wherein objects with anegative involvement level higher than a negative involvement levelthreshold are excluded for a selection of objects.
 2. The method ofclaim 1, wherein category characteristics of objects are clustered andthe at least one category characteristic of interest is determined byselecting at least one cluster of category characteristics representingthe at least one category characteristic of interest.
 3. The method ofclaim 1, wherein at least two different types of gaze interaction eventsare determined for an object.
 4. The method of claim 1 wherein theperipheral view is modeled with an area surrounding a focus point andexcluding the focus point.
 5. The method of claim 1, wherein categorycharacteristics for at least two different types of object categoriesare determined for an object.
 6. The method of claim 1, wherein: thecategory characteristics for the collection of objects are precalculatedand stored together with the objects; and/or wherein the categorycharacteristics are determined by evaluating object data of the objects.7. The method of claim 1, wherein object categories are determined byevaluating object data of objects from the group of interesting objects.8. The method of claim 1, further comprising: determining an involvementlevel for an object from the gaze interaction events of the object,wherein only objects with an involvement level higher than aninvolvement level threshold are considered for the selection of objects.9. (canceled)
 10. The method of claim 1, wherein: the collection ofobjects is displayed on a display device in the field of view of theperson; and/or after the at least one category characteristic ofinterest has been determined, additional objects having the at least onecharacteristic of interest are displayed on the display device.
 11. Themethod of claim 1, wherein: the map of objects is generated fromoutlines of objects; and/or the map of objects is generated from imagedata received from a camera recording the field of view of the person.12. The method of claim 1, wherein: the collection of objects comprisesat least one of: images, words, text blocks, icons, formulas, pages,sections of a document, 3D objects, shapes, patterns; and/or the objectcategories are content descriptors; and/or the category characteristicsare values of content descriptors.
 13. (canceled)
 14. A non-volatilecomputer-readable medium, in which a computer program for detectingobjects of interest is stored, which, when being executed by aprocessor, is adapted to carry out the steps of: receiving gaze trackingdata of a person looking at a collection of objects, the gaze trackingdata comprising information, at which points the person is looking;determining a map of objects in the field of view of the person, the mapof objects indicating, at which position an object in the collection ofobjects is arranged; determining gaze interaction events for objects inthe collection of objects by matching the gaze tracking data to the mapof objects, wherein a gaze interaction event for an object indicatesthat a specific gazing interaction has occurred with respect to theobject and wherein a gaze interaction event is based on a model of aperipheral view of the person; determining category characteristics forobjects having gaze interaction events, wherein a categorycharacteristic describes the object with respect to a specific objectcategory; determining at least one category characteristic of interestin at least one object category by finding at least one categorycharacteristic which is assigned to a plurality of objects, which havegaze interaction events indicating the person is interested in theseobjects; determining a negative involvement level for an object from thegaze interaction events of the object based on the model of theperipheral view of the person, wherein objects with a negativeinvolvement level higher than a negative involvement level threshold areexcluded for a selection of objects.
 15. A gaze tracking system,comprising: a gaze tracking device for generating gaze tracking data;and an object detection device adapted for performing the steps of:receiving gaze tracking data of a person looking at a collection ofobjects, the gaze tracking data comprising information, at which pointsthe person is looking; determining a map of objects in the field of viewof the person, the map of objects indicating, at which position anobject in the collection of objects is arranged; determining gazeinteraction events for objects in the collection of objects by matchingthe gaze tracking data to the map of objects, wherein a gaze interactionevent for an object indicates that a specific gazing interaction hasoccurred with respect to the object and wherein a gaze interaction eventis based on a model of a peripheral view of the person; determiningcategory characteristics for objects having gaze interaction events,wherein a category characteristic describes the object with respect to aspecific object category; determining at least one categorycharacteristic of interest in at least one object category by finding atleast one category characteristic which is assigned to a plurality ofobjects, which have gaze interaction events indicating the person isinterested in these objects; determining a negative involvement levelfor an object from the gaze interaction events of the object based onthe model of the peripheral view of the person, wherein objects with anegative involvement level higher than a negative involvement levelthreshold are excluded for a selection of objects.